Endoscopic video imaging system with automatic reorientation

ABSTRACT

A video imaging system is provided. The video imaging system is configured to maintain a video image displayed on a display unit in a predetermined orientation based upon a use. The video imaging system includes an endoscope, a camera head unit, a camera control unit configured to determine a focal distance, and a display control unit. The display control unit is configured to process the focal distance to determine a use. The use is processed by the display control unit so as to maintain the video image in the predetermined orientation. The contextual information may be focal distance, image data or endoscopic orientation.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/375,974 filed on Jul. 14, 2021, and entitled, “Automatic ImageOrientation Based on Use,” which is in turn a continuation of U.S.application Ser. No. 16/665,706 filed on Oct. 28, 2019, titled“Automatic Image Orientation Based on Use” and issued as U.S. Pat. No.11,070,745 on Jul. 20, 2021; the entire contents all above namedapplications are incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates to a camera configured to automatically orientan image based upon use

BACKGROUND

Video cameras provide a video image based upon the orientation of thecamera. In some embodiments, the physical dimension of the cameraprovides the user with a visual indicator to ensure the video image isoriented in a desired orientation. For instance, it is easy to capture avideo image from a cuboidal shaped video camera in a predeterminedorientation as it is easily recognizable when the camera is upright.However, in certain applications, such as an endoscope, it is difficultto determine when the endoscope is upright as the endoscope is generallycylindrical. Further, the orientation of the video image may rotate whenthe endoscope is handed from one person or another, such as from thenurse to the surgeon.

An example will be described with reference to FIGS. 1A-2B. FIG. 1Adepicts a surgical procedure with a nurse 100 and a doctor 200 standingover a patient 300. FIG. 1A shows the nurse 100 holding the endoscope22. The nurse 100 turns on the endoscope 22 and currently levels theendoscope 22. As used herein, the term level means rotating the videoimage 400 on the display to a desired orientation. The nurse 100 mayrotate the grip of the endoscope 22 to orient the imager in an uprightorientation so that outside of the surgical site, the video image isaligned with the orientation of the physical space. That is, the doorsand walls of the video image 400 are upright as is seen by the nurse 100and the surgeon 200, as shown in FIG. 1B. Alternatively, an actuator,such as a button, may be pressed to rotate the video image 400.

FIG. 2A is a depiction of FIG. 1A after the nurse 100 has handed theendoscope to the surgeon 200. When the endoscope 22 is handed over, thesurgeon 200 may grip the endoscope 22 in a different orientation, whichrotates the video image, as shown in FIG. 2B. Thus, the surgeon 200 hasto rotate the endoscope 22 within his/her grip to orient the video image400 to align with the physical orientation of the room or press theactuator as the case may be so as to level the video image 400 to adesired orientation, or the orientation shown in FIG. 1B.

Further, the orientation of the video image 400 of the surgical site mayrotate as the surgeon 200 manipulates the endoscope within the surgicalsite. Such a change in the video image 400 requires the surgeon 200 tocorrelate what is seen on the display with the actual position of theendoscope 22.

Accordingly, it remains desirable to have an imaging system configuredto automatically orient the video image to a desired orientation.

SUMMARY

One aspect of the disclosure provides an imaging system configured toorient an image based upon use of the imagine system. The imaging systemincludes a camera head unit, a light source and a display control unit.The camera head unit includes a camera head and a camera control unit.The camera head unit is configured to obtain image data and process theimage data into an image signal. The camera head includes an imagesensor. A light source generates electromagnetic radiation which iscaptured by the image sensor in the form of image data.

The camera head may be coupled to an endoscope. A display control unitis further configured to process the focal distance to determine a useand process the image so as to display the image onto a display unit inan orientation associated with the determined use. The use may be asurgical use or a non-surgical use. In some implementations, the imagingsystem is a video imaging system, such that the image signal is a videosignal.

In some aspects of the imaging system, the imaging system includes anorientation unit is configured to detect at least one of an accelerationand a gravitational relationship of the endoscope. The orientation unitmay be disposed in the camera head unit. The orientation unit mayinclude an accelerometer or a gyroscope sensor. The orientation unitdetects an acceleration and/or a gravitational relationship of thecamera and transmits the detected acceleration and/or gravitationalrelationship to the display control unit. The display control unitprocesses the acceleration and/or gravitational relationship todetermine a use.

In another aspect of the imaging system, the display control unitprocesses an acceleration and/or a gravitational relationship along witha focal distance to determine a use.

In another aspect of the imaging system, additional contextualinformation is provided to further identify the type of surgical use.The imaging system may include a database storing a plurality ofsurgical procedures. The surgical procedures may include contextualinformation relating to the corresponding procedure to include imagedata and ranges for endoscopic orientation. Each surgical procedure isalso associated with a predetermined image orientation. Thepredetermined image orientation may include a predetermined video imageorientation.

Image data may include anatomical features such as a meniscus, a labrumor the like. The endoscopic orientation may be determined by theprocessing information gathered by the orientation unit. The contextualinformation may be processed by the display control unit to determinewhich surgical procedure in the database is being performed and toprocess the image (e.g., the video image) so as orient, maintain anddisplay the image in the predetermined image orientation.

In another aspect of the present disclosure, the imaging system includesan input. The input is configured to select any one of the plurality ofsurgical procedures stored in the database. In such an embodiment, thedisplay control unit automatically orients the image signal (e.g., thevideo signal) so as to maintain the image (e.g., the video image) in anorientation corresponding to the selected surgical procedure.

In another aspect of the present disclosure, the display control unitmay be programmed with a neural network for learning.

In another aspect of the present disclosure, the video imaging systemmay further include a second input configured to adjust the orientationof the image (e.g., the video image) displayed on the display unit.

In yet another aspect of the present disclosure, a method for displayinga video image onto a display is provided. The method includesdetermining, with a camera control unit, a focal distance. The methodalso includes processing, with a display control unit, the focaldistance so as to determine a use. The method further includesprocessing, with the display control unit, the use to maintain the videoimage in a predetermined orientation.

This aspect may include one or more of the following features. In someimplementations, a camera head unit includes a lens and an image sensor.The method may include moving the lens relative to the image sensor. Themethod may also include controlling, with the camera head unit, adisplacement of the image sensor relative to the lens so as to focus thevideo image when there is a change in the focal distance.

In some implementations, the method includes detecting, with anorientation unit, at least one of an acceleration and a gravitationalrelationship of an endoscope. The method may also include processing,with the display control unit, the at least one of the acceleration andthe gravitational relationship of the endoscope so as to determine theuse.

In some implementations, the method includes storing, in a database, aplurality of surgical procedures, each of the plurality of surgicalprocedures having a video predetermined orientation, and each of theplurality of surgical procedures being a surgical use.

In some implementations, each of the plurality of surgical proceduresstored in the database includes contextual information including animage data.

In some implementations, the method includes processing, with thedisplay control unit, image data with the video image. The method mayalso include determining, with the display control unit, which of thesurgical procedures of the plurality of surgical procedures is beingperformed. The method may further include processing, with the displaycontrol unit, the video image so as to maintain the video image in thepredetermined video orientation.

In some implementations, the method includes processing, with the camerahead unit, a range for an endoscopic orientation. The method may alsoinclude processing, with the camera head unit, the at least one of theacceleration and the gravitational relationship of the endoscope so asto determine an endoscopic orientation. The method may further includeprocessing, with the display control unit, the endoscopic orientation todetermine a surgical procedure.

The details of one or more implementations of the disclosure are setforth in the accompanying drawings and the description below. Otheraspects, features, and advantages will be apparent from the descriptionand drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1A is a depiction showing a nurse holding an endoscope in a surgeryroom;

FIG. 1B is a depiction of an image of the endoscope shown in FIG. 1A;

FIG. 2A is a depiction showing a surgeon holding the endoscope;

FIG. 2B is a depiction of an image of the endoscope shown in FIG. 2A;

FIG. 3 is a schematic view of a camera according to the principles ofthe present disclosure;

FIG. 4A is a depiction of an image having a long focal distance;

FIG. 4B is a depiction of FIG. 4A showing the image angled;

FIG. 4C is a depiction of the prior art method of manually leveling theimage shown in FIG. 4B;

FIG. 4D is a depiction showing the image shown in FIG. 4B beingmaintained in a predetermined orientation;

FIG. 5A is a depiction of an image having a short focal distance;

FIG. 5B is a depiction of FIG. 5A showing the image angled; and

FIG. 5C is a depiction showing the image shown in FIG. 5B beingmaintained in a predetermined orientation.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

A video camera configured to automatically orient a video image basedupon a determined use of the video camera is provided. A display controlunit processes contextual information such as focal distance, image dataand endoscopic orientation to determine a use. The use is processed bythe computer processing unit so to maintain the video image in apredetermined orientation. Accordingly, the video camera automaticallyorients the video image based upon the determined use.

With reference now to FIG. 3 , an illustrative depiction of a videoimaging system 10 is provided. The video imaging system 10 includes acamera head unit 12, a light source 14 and a display control unit 16.The camera head unit 12 includes a camera head 18 and a camera controlunit 20. The camera head unit 12 is configured to obtain image data andprocess the image data into a video signal, as explained in greaterdetail below. The camera head 18 is coupled to an endoscope 22 by opticsincluding a plurality of lenses 24. The camera head 18 includes an imagesensor 26. For illustrative purposes, a description of the video imagingsystem 10 is provided within the context of an endoscope 22. However, itshould be appreciated that other scopes may be used, illustrativelyincluding an exoscope, borescopes and the like. The image sensor 26 maybe disposed within the endoscope 22, for example, adjacent to or inplace of the lens 24 as in a video endoscope. Similarly, while a videoimaging system 10 is generally shown and described herein, it should beappreciated that other imaging systems (e.g., still imaging systemshaving still cameras capturing still images) may be used.

The light source 14 generates electromagnetic radiation which iscaptured by the image sensor 26 in the form of image data. The lightsource 14 may be formed by one or more Light Emitting Diodes (LED)within an external light source housing providing “cold light”transmitted via optical fiber connection to the endoscope 22 as is knownin the art.

The camera head 18 is in communication with the camera control unit 20through either a cable or a wireless connection. The camera control unit20 controls various processing functions of the camera head unit 12 toinclude timing of the light source 14, readout of the image sensor 26and the like. The image sensor 26 functions in coordination with thelight source 14 to gather image data which is processed by the cameracontrol unit 20 to output a video signal. The video signal is processedby the display control unit 16. The display control unit 16 processesthe video signal for display onto a display unit 28. The endoscope 22and camera head 18 may form an integral unit or may be detached fromeach other as singular functional pieces. Regardless of the specificconfiguration, the principles of the present disclosure apply to variousexamples of video imaging systems 10.

The image sensor 26 can be a complementary metal oxide semiconductor“CMOS” or a charged coupled device “CCD”. It should be appreciated thatany pixelated image sensor 26 currently known or later developed may bemodified and adopted for use herein. In one embodiment, the image sensor26 is configured to receive electromagnetic radiation in the visiblespectrum and also in the infrared range between about 800 nanometers to1200.

In another aspect, the video imaging system 10 may include a pair ofimage sensors 26, wherein one of the image sensors 26 is configured toreceive electromagnetic radiation in the visible spectrum with aparticular field of view and the other of the image sensors 26 isconfigured to receive electromagnetic radiation in the infrared rangebetween about 800 nanometers to 1200 nanometers. One skilled in the artwould recognize that various systems using combinations of one or moreimage sensors 26 may benefit from the principles of the presentdisclosure. The endoscope 22 outputs the video image onto a display unit28.

In one aspect of the video imaging system 10, a focal distance of thecamera unit may be processed by the display control unit 16 to determinea use. Alternatively, the display control unit 16 may process the focaldistance along with other contextual information so as to determine theuse. As used herein, the term “focal distance” refers to a distance thatan object appears to be from a lens (e.g., lens 24) when in focus. Forillustrative purposes, a short focal distance shall refer to a videoimage of an object 10 cm or less from the lens 24 whereas a long focaldistance refers to a video image of an object farther than 10 cm. Itshould be appreciated that the provided length of a short focal distanceand long focal distance is exemplary and is not limiting to the scope ofthe appended claims.

The spatial displacement may be achieved manually or automatically. Forinstance, in a camera head unit 12 with a manual focus, the spatialdisplacement may be determined based upon the displacement of the imagesensor 26 with respect to a fixed lens 24, a displacement of a lens 24with respect to a fixed image sensor 26 or a displacement of both amoveable lens 24 and a moveable image sensor 26. Displacement of thelens 24, image sensor 26 or both may be achieved using known mechanicaland electro-mechanical actuators and drives which may be controlled by adial or a button disposed on the endoscope 22 or automatically in thecase of auto-focus. Signals from the dial or button, as the case may be,may be processed by the camera control unit 20 to move the lens 24and/or the image sensor 26. The adjustment processed by the cameracontrol unit 20 may be transmitted to the display control unit 16 todetermine the use.

For example, the display control unit 16 may determine that theendoscope is in a non-surgical use when the distance between the lens 24and the image sensor 26 is at its shortest distance, placing the focusat infinity and thus, focal distance is long. Alternatively, the displaycontrol unit 16 may determine that the camera is in a surgical use whenthe distance between the lens 24 and the image sensor 26 is at itsgreatest distance, and the focal distance is short. It should beappreciated that the display control unit 16 may be programmed todetermine a long or short focal distance based upon a range of distancebetween the lens 24 and the image sensor 26. For illustrative purposes,a long focal distance is commensurate with the endoscope 22 operatingoutside of the body and is determined by a distance between the lens 24and the image sensor 26 that is below a predetermined distance. A shortfocal distance is commensurate with the endoscope 22 operating inside ofthe body and is determined by a distance between the lens 24 and theimage sensor 26 that is greater than or equal to the predetermineddistance.

As used herein, non-surgical use refers to an instance where theendoscope 22 is operating outside of the body. In such an instance, thedisplay control unit 16 processes the video signal such that the videoimage displayed on the display unit 28 is oriented in an uprightposition. As used herein, upright refers to a vertical axis with respectto the horizon. Thus, the video imaging system 10 may use the focaldistance to automatically orient the video image displayed on thedisplay unit 28 in an upright position which allows for (i) rotation ofthe endoscope 22 as it is passed outside of the patient's body from oneuser to the other, or (ii) rotation of the endoscope 22 by a singleuser. In another example, the camera head unit 12 may be equipped withan automatic zoom function, wherein displacement of lens 24 with respectto image sensor 26 is made automatically.

The video imaging system 10 may further include an orientation unit 30.The orientation unit 30 may be disposed in the camera head unit 12. Theorientation unit 30 is configured to detect at least one of anacceleration and a gravitational relationship of the endoscope 22. Forexample, the orientation unit 30 may include an accelerometer 32. Anyaccelerometer 32 currently known and/or or later developed may beadapted for use herein, illustratively including a device commonly knownas a micromachined microelectromechanical system. The accelerometer 32may be mounted to the camera head unit 12 and may detect an accelerationof the camera made in reference to the Earth's gravity.

In another example of an orientation unit 30, the orientation unit 30includes a gyroscope sensor 34 configured to provide a gravitationalrelationship of the endoscope 22. In particular, the gravitationalrelationship includes the orientation and angular velocity of the camerahead unit 12. Preferably, the gyroscope sensor 34 is one of a pluralityof electric devices currently known and/or later developed andconfigured to detect orientation and angular velocity, illustrativelyincluding solid state ring lasers, a fiber optic gyroscope and/or aquantum gyroscope. The gyroscope sensor 34 may be mounted to the camerahead unit 12 and may detect an acceleration of the camera made inreference to the Earth's gravity. In another embodiment, the orientationunit 30 includes both an accelerometer 32 and a gyroscope sensor 34.Alternatively, the orientation unit 30 may include both an accelerometer32 and a gyroscope sensor 34.

The orientation unit 30 detects an acceleration and/or a gravitationalrelationship of the camera and transmits the detected accelerationand/or gravitational relationship to the display control unit 16. Thedisplay control unit 16 processes the acceleration and/or gravitationalrelationship to determine a use. For example, based upon theacceleration or orientation of the camera head unit 12 and/or theendoscope 22, the display control unit 16 may determine that theendoscope 22 is out of the body, and thus determines a non-surgical use,wherein the video image is oriented and maintained in an uprightposition on the display unit 28. Thus, the video imaging system 10 mayuse acceleration and/or a gravitational relationship of the camera headunit 12 and/or endoscope 22 to automatically orient the video image inan upright position which allows for rotation of the endoscope 22 as itis being used outside of the body.

In another aspect of the video imaging system 10, the display controlunit 16 processes an acceleration and/or a gravitational relationshipalong with a focal distance to determine a use. For example, a shortfocal distance and an acceleration below a predetermined accelerationmay be processed by the display control unit 16 to determine a surgicaluse. As used herein, the term “surgical use” refers to the operation ofthe endoscope 22 within a body. For example, a distance between the lens24 and an image sensor 26 which is greater than the predetermineddistance provides a magnified view, indicating that the object of theimage is near. Thus, the display control unit 16 makes a determinationthat the endoscope 22 is within the body and a surgical use isdetermined.

With reference again to FIG. 3 , an aspect of the video imaging system10 is provided wherein additional contextual information is provided tofurther identify the type of surgical use. The video imaging system 10includes a database stored in memory 36. The database 36 may be storedin the display control unit 16. Any database 36 architecture currentlyknown or later developed may be modified for use herein, illustrativelyincluding a computer program unit architecture. The database 36 isconfigured to store a plurality of surgical procedures, each of thesurgical procedures being a surgical use.

The surgical procedures stored in the database 36 may include contextualinformation relating to the corresponding procedure to include imagedata and ranges for endoscopic orientation. Each surgical procedure isalso associated with a predetermined video image orientation. Forinstance, a video image displayed on a display unit 28 for a procedureto repair a torn meniscus may be oriented so as to maintain the meniscusalong a horizontal axis as described in greater detail below.

Image data may include anatomical features such as a meniscus, a labrumor the like. Image data may also include medical devices such asimplants such as a screw, a cage or the like and medical instruments.The display control unit 16 is further configured to process the videoimage to determine which of the surgical procedures stored in thedatabase 36 is being performed. The display control unit 16 may befurther configured to process the video signal so as to maintain thevideo image in a predetermined video image orientation.

For example, as the endoscope 22 is inserted into a body cavity, thefocal distance of the camera head 18 is short, thus the spatial distancebetween the lens 24 and the image sensor 26 is greater than apredetermined distance, e.g., longer than the lens focal length. Thedisplay control unit 16 may determine that the endoscope 22 is beingused in a surgical use based solely upon the short focal distance or mayprocess the short focal distance along with contextual information fromthe orientation unit 30 to determine that the endoscope 22 is in asurgical use.

Once, the display control unit 16 determines that the endoscope 22 is ina surgical use, the display control unit 16 may process additionalcontextual information so as to determine which of the plurality ofsurgical procedures stored in the database 36 is being performed. Forexample, the display control unit 16 may determine a surgical procedureby processing image data to identify an anatomical feature or a medicaldevice, or by processing the endoscopic orientation. The endoscopicorientation may be determined by the camera head unit 12 by processingthe acceleration and/or gravitational relationship of the endoscope 22.It should be appreciated that the display control unit 16 may beprogrammed to make a determination of a surgical procedure based uponany singular contextual information, e.g., image data or endoscopicorientation of the endoscope 22 or a combination of contextualinformation.

In another aspect of the present disclosure, the video imaging system 10includes an input 38. The input 38 may be disposed on the head unit, orthe display control unit 16. The input 38 is configured to select anyone of the plurality of surgical procedures stored in the database 36.In such an embodiment, the display control unit 16 automatically orientsthe video signal so as to maintain the video image in an orientationcorresponding to the selected surgical procedure.

The display control unit 16 may be programmed with a neural network 40for learning, wherein the display control unit 16 processes the selectedsurgical procedure with the contextual information received from thecamera head unit 12 to determine if the selected surgical procedure isthe same as the surgical procedure determined by the display controlunit 16 based off of the contextual information. The database 36 may beupdated when a determined surgical procedure is different than aselected surgical procedure.

In another aspect of the present disclosure, the video imaging system 10includes a second input 42. The second input 42 may be disposed on thehead unit, or the display control unit 16. The second input 42 isconfigured to adjust the orientation of the video image displayed on thedisplay unit 28. This allows the surgeon to set the orientation of thevideo image to his/her preference. The deviation between what thesurgeon selected and what was processed by the display control unit 16may be stored by the memory 36 and analyzed by the neural network 40 soas to adjust the video image for the same type of surgical procedureperformed in the future.

With reference now to FIGS. 4A-4D an illustration of an operation of thevideo imaging system 10 using a focal distance is provided. FIG. 4Aprovides a video image of the endoscope 22 when the endoscope 22 is heldso as to retain the video image in an upright position. FIG. 4Aillustrates the upright position. FIG. 4B illustrates an example of aninstance, where the nurse holds the endoscope 22 and turns on theendoscope 22. The corresponding video image shows how the nurse isholding the endoscope 22 in a manner which produces a video image whichis turned relative to an upright position as indicated by the arrow. Inparticular, FIG. 4B shows the door being angled. FIG. 4C is a depictionshowing the prior art method where, in particular, the nurse has torotate the endoscope 22 so as to rotate the video image into an uprightmanner.

As described above, the video imaging system 10 is configured to processa focal distance to automatically orient and maintain the video image inan upright manner. The video image in FIGS. 4A-4D show an example of along focal distance, as indicated by the focus image of the door. FIG.4D shows the door and equipment being oriented in the same field of viewas seen by a person standing upright in the room. The bi-directionalarrow beneath the endoscope 22, shown in FIG. 4D, indicates a rotationof the endoscope 22. However, the video image remains upright. Thus, byprocessing at least a focal distance, the video imaging systemdetermines a non-surgical use and the video image on the display unit 28is maintained in an upright manner regardless of the rotation of theendoscope 22.

As described above, the focal distance may be determined by the apertureand the lens focal length. The lens focal length may be determined by aspatial displacement between the lens 24 and the image sensor 26. Thevideo image of FIG. 4D shows the focus is set to infinity, indicatingthat the spatial distance between the lens 24 and image sensor 26 isspaced apart from each other by less than, or equal to, a predetermineddistance. The display control unit 16 may process the distance betweenthe lens 24 and the image sensor 26 based upon the actual distance whichmay be detected by a sensor such as a transducer mounted to the movingelement, or by processing the position of the dial or actuation of abutton which controls the focus. The display control unit 16 processesthe spatial distance so as to determine that the endoscope 22 is in anon-surgical use and processes the video signal so as to maintain thevideo image in an upright manner. Thus, as the endoscope 22 is rotated,as indicated by the bi-directional arrow in FIG. 4D, the video imageremains upright on the display. As described above, the rotation may bedone by an individual, or when endoscope 22 is handed over to thesurgeon.

The video imaging system 10 may process other contextual informationgathered by the orientation unit 30 to determine the use. For instance,when the orientation unit 30 determines that the endoscope 22 is subjectto a large acceleration which is not commensurate with surgical use, butdetermines that the focal distance is short, the display control unit 16determines that the endoscope 22 is in a non-surgical use. Such ascenario may be commensurate with the nurse or surgeon cleaning the lens24.

With reference now to FIGS. 5A-5C an illustration of an operation of thevideo imaging system 10 using other contextual imaging to orient thevideo image is provided. FIGS. 5A-5C show an illustration of a shortfocal distance. For illustrative purposes, the illustration is providedin the context of a procedure to correct a meniscus. FIG. 5A shows themeniscus in a preferred orientation wherein the longitudinal length,indicated by line “L-L” of the meniscus is generally aligned with thehorizon.

FIG. 5B shows an example of the video image taken by the endoscope 22that is made without any correction or leveling. This may result fromthe difference between how the video imaging was leveled to an uprightposition by the display control unit 16 when a non-surgical use wasdetermined. As the endoscope 22 is inserted into the body, the focaldistance is changed from a long focal distance to a short focaldistance. As such, the distance between the lens 24 and the image sensor26 is equal to or greater than the predetermined distance. The displaycontrol unit 16 process the distance between the lens 24 and the imagesensor 26 to determine that the endoscope 22 is in a surgical procedure.

The video imaging system 10 is configured to process additionalcontextual information to determine what type of surgical use, e.g., thetype of surgical procedure being performed. The contextual informationthat may be processed includes image data and/or ranges for endoscopicorientation. For instance, the approach angle of the endoscope 22 may bedetermined by the orientation unit 30 and may be processed inconjunction with image data to determine the type of surgical procedurethat is being performed. For illustrative purposes, FIGS. 5B and 5C willbe described wherein image data is processed by the display control unit16 to determine the type of surgical procedure being performed.

As described above, the database 36 stores a plurality of surgicalprocedures. The surgical procedures may include contextual informationrelating to the corresponding procedure to include image data and rangesfor endoscopic orientation. In this case, the image of a meniscus isassociated with one of a plurality of medical procedures to include aprocedure to correct a radial tear, a horizontal tear, flap tear or thelike. Each of these tears may also include a stored range of endoscopicorientations, wherein the display control unit 16 processes the videoimage to identify a meniscus. The type of the meniscus procedure may bedetermined based upon the endoscopic orientation, wherein the actualorientation of the endoscope 22 may also be processed by the displaycontrol unit 16 to confirm which of the types of meniscus procedures isbeing performed. The actual orientation of the endoscope 22 may bedetermined by information gathered by the orientation unit 30.

Once the display control unit 16 identifies the meniscus, the videoimage is leveled, or oriented, to a predetermined orientation, as shownin FIG. 5C. FIG. 5C also depicts how the video image remains leveledeven when the endoscope 22 is rotated. It should be appreciated thatother image data, such as a medical implant or a tool may be stored inthe database 36 and processed by the display control unit 16 todetermine the type of surgical procedure being performed, which alsolevels and maintains the video image in accordance with the orientationassociated with said medical procedure.

The video imaging system 10 may include an input 38. The input 38 isconfigured to select any one of the plurality of surgical proceduresstored in the database 36. In such an embodiment, the display controlunit 16 automatically orients the video signal so as to maintain thevideo image in an orientation corresponding to the selected surgicalprocedure even when the endoscope 22 is rotated, as illustratively shownin FIG. 5C. The video imaging shown on the display unit 28 may bemanually rotated by the user by actuation of a second input in instanceswhere the surgeon prefers an orientation different than what was leveledby the display control unit 16.

As described above, the display control unit 16 may be programmed with aneural network 40 for learning. In some implementations, the displaycontrol unit 16 processes the selected surgical procedure with thecontextual information received from the camera head unit 12 todetermine if the selected surgical procedure is the same as the surgicalprocedure determined by the display control unit 16 based off of thecontextual information. In particular, the neural network 40 may processa selected surgical procedure (e.g., a surgical procedure selected bythe input 38 from the plurality of surgical procedures stored in thedatabase 36) with a determined surgical procedure (e.g., the surgicalprocedure determined by the display control unit 16 based off of thecontextual information) and update the contextual information in thedatabase 36 with actual contextual information of the selected surgicalprocedure when the determined surgical procedure is different than theselected surgical procedure.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

1. A video imaging system operable to display a video image onto adisplay, the video imaging system comprising: an endoscope; a camerahead unit; an image processor; a focal distance processor; a display; acontrol unit configured to determine, based a focal distance determinedby the focal distance processor, a mode of operation of the endoscope,and adjust a view orientation displayed on the display accordingly.
 2. Amethod for displaying a video image on a display, comprising the stepsof: determining, with a focal distance determining unit, a focaldistance; processing, with a display control unit, the focal distance soas to determine a mode of operation; displaying, with the displaycontrol unit, a video image in a predetermined orientation when the modeof operation matches a predetermined value.